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ABSTRACT 

As part of the evaluation of the College of Education 
at the University o.f Tennessee, an annual survey is conducted to 
obtain principals' ratings of graduates who are certified beginning 
teachers. This survey has enjoyed a response rate in excess of 90 
percent. The survey is conducted by mail/ with one mail followup, .and 
occasionel telephone followups. The first step is to identify the 
specific school locations of the teaching graduates. Then the survey 
forms and cover letters are mailed to the principals. One followup 
letter and another copy of the quest ionnad re are sent to those 15 to 
29 percent who have not responded after four, weeks. Telephone calls 
are also made, but such calls have pot exceeded three in any. of the ' 
three years discussed. Five different introductory cover letters are 
used, as well as personal individualized letters when indicated. 
Return envelopes are provided, and the le.tters are prepared on high 
quality letterhead stationery. These procedures are satisfactory in 
producing a^high response rate. The three-page questionnaire, which 
is appended, includes demographic information, a rating scale of 
teacher effectiveness, and a check list of factors which influenced 
the principal to hire this teacher. (GDC) 
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A Consistently Successful Follow-up Survey 

In doing a survey one major objective is to achieve a high response 
rate so that the researcher can feel the data are representative and not 
subject to nonresponse bias. The survey that is being described is consid- 
ered a success from this standpoint because it has generated response rates 
in excess of 907. each time it has been conducted during each of the past 
three years. In addition, these response rates were produced using a mail 
survey with one mail follow-up and, in a very few instances, a telephone 
follow-up. 

It has not been possible to identify apy one feature or procedure as 
being responsible for the high response rate. With a satisfactory result 
there has been no interest in randomly assigning half of those surveyed to 
a control group which might possibly achieve less positive results or in 
varying the procedures used. This is most accurately a case study or a care- 
ful description of the procedures used. 

Some of the procedures used are typical of Dillman's Total Design Method 
(1978), but others are not. Some are among those identified by Altschuld 
and Lower (1984) as elements contributing to their high return rate when 
they surveyed school personnel regarding perceptions and attitudes toward 
evaluation of teaching in general. Some of the procedures are also consist- 
ent with recent research on response rates (Baumgartner 6c Heberlein, 1984; 
Yu 6c Cooper, 1983). 

Purpose 

The purpose of the annual survey was to obtain ratings of the perform- 
ance of graduates of the College of Education from principals of public school 



in the state under whose supervision they worked during the year following 
graduation. The survey is part of the on-going program evaluation efforts 
of the College of Education. It has been conducted using^^Hie^T^edures 
described herein for the past three years. This is in no way intended to 
be an evaluation of individuals (graduates) or of the schools or school sys- 
terns in which they are employed. This survey provides dafa to supplement 
that resulting from surveying a stratified (by college major) random sample 
of all graduates each fall to gain demographic and 5 * employment information 
and for purposes of program evaluation. 

Procedures 

Identifying the Population 

The first step in the process is to locate graduates who were teaching. 
Using their computerized records, the State Department of Education is able 
to: 1. identify those teachers who have graduated from the institution and 
who have received certification during the specified time period (certifica- 
tion is granted only upon completion of degree and meeting certification 
requirements) which begins just after the graduation date of fall quarfer 
and continues until jusc prior to the graduation date of the fall quarter 
of the following year; 2. locate the public school systems in which any of 
those teachers a^e reported as employed on the fall reports filed by all 
public school systems in the state. Checking the list thus prepared by the 
State Department of Education against University graduation lists makes it 
possible to delete from the list any teachers who have received initial certi- 
fication during the specified time period but tfho have been postbaccalaureate 
students or who have returned only to complete coursework for teaching after 
receiving a degree elsewhere* 



From 90 to 100 graduates were teaching in 36 to 38 public schools within 
the state in the fall following their graduation. The city school system 
and the county school system in the county in which the University is located 
employed from 297. to 347. of those teaching graduates. 
Timelines 

The first step of the process, requesting the list from the State Department 
of Education, is initiated in late January. The information is not ordinarily 
available any earlier because of the time involved in updating the computer 
records each year. When the listing of teachers by school systems is received, 
a letter is sent to the two local school systems (which employ the largest 
numbers of graduates), requesting specific school assignments for those teach- 
ers in their respective systems. Both school systems have been very coopera- 
tive in supplying the names of specific schools to which each of the teachers 
employed by them has been assigned. Accompanying the request letter and 
the list of teachers from their system is a copy of the questionnaire which 
will be sent to the principals in the system. The request for school assign- 
ments is usually made in late February or early March, as soon as the infor- 5 ^ 
mation is received from the State Department of Education. 

The survey forms and cover letters -are sent in early or mid-April (April 
10-19). If Easter occurs in early April, the surveys are not sent until 
after the schools are back in session. One follow-up letter and another 
cofy of the questionnaire are sent after approximately four weeks to those 
who have not responded. Since some school systems conclude the school year 
before the end of May it is important that the follow-up arrive at least 
one week before schools close. In the, case of the local city and county 
systems in which questionnaires are sent directly to the principals, it is 
possible to initiate a telephone call if a respon&e has still not been received 
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after the follow-up mailing. Such calls have not exceeded three in any of 
the three years which form the basis of this paper. 
Personal izat ion 

^ A variety of cover letters is used with the questionnaire. When only 
the school system is known a letter explaining the purpose of the survey 
is personally addressed to the superintendent. The principal's cover letter 
questionnaire, and business reply envelope are enclosed in an unsealed en- 
velope with, "To the principal of Jane Doe, Teacher Number 12304567" tvped 
on the front of it. The envelope is thus ready for the superintendent to 
route to the appropriate principal. It is also unsealed so that the super- 
intendent is free to examine its contents. 

For those teachers who are employed in the local city and county sys- 
tems for whom the specific school assignments have been determined, question 
naires, business reply envelopes, and cover letters are sent directly to 
the principals. A separate cover letter, questionnaire, and reply envelope 
are included for each graduate if more than one are employed in a system 
or school. A manilla mailing envelope of appropriate size is used for all 
materials for a ^school or system if two or more graduates are employed. 

Another facet of personalization occurs in the body of the cover let- 
ter in reference to the number of graduates. Basically, there are five in- 
troductory form letters used: 

1. explanatory letter to superintendent employing only one graduate 

2. explanatory letter to superintendent employing more than one grad- 
uates 

3. cover letter to unknown principal ^ 

4. cover letter to known principal having one teacher 

5. cover letter to known principal having two or more teachers 



The relatively small number of nonrespondents after the first mailing 
and an even greater number of variations makes it more efficient in some 
cases to type the follow-up letters to fit**the following circumstances: 
; 1. superintendent employing only one graduate 

2. superintendent employing more than one graduate, no forms returned 

A - 

3. superintendent employing more than one graduate, one or more forms 
returned 

4. unknown principal 

5.. known principal having one graduate 

c 

. 6. known principal having more than one graduate", no forms returned 
7. known principal having more than one graduate, one or more forms 
returned 

Cover letters to the superintendents include reference to State Department 
of Education records as the source of the information, regarding the employ- 
ment location of the graduates. In letters to the principals in the two 
local school systems it is acknowledged that the graduates were located with 
the cooperation of their respective school systems. Tacit approval of the 
survey is implied through the cooperation of the State Department and the 
school superintendents w.ho either identify the specific ^chools (in the two 
local systems) or direct the questionnaires to the principals without identi- 
fying either the principals or schools to the College. 

Cover letters are duplicated on letterhead stationery of bond quality. 
Although the body of the cover letters is duplicated, the inside address 
and greeting are typed as are the names and addresses on the envelopes. 
The signature on the form letter (that of the director of the Bureau of Educational 
Research and Service) is signed before duplication. 
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Postage 

Questionnaires are mailed by first-class postage, using the University 
postage meter rather than stamps. Printed business reply envelopes are in- 
cluded in the initial mailing but not in the follow-up mailing. 

c 

Questionnaires 

The questionnaire has been two or three pages in length and duplicated 
on colored paper. The first page contains basically demographic information 
(see Appendix). The second page consists of 14 rating scales on which the"" 
principal assesses the performance cf the teacher, with the scales correspond- 
ing to the rating scales used by cooperating teachers and University coordina- 
tors during student teaching. Eact? of the 14 rating scales can be completed 
by circling a number from 1 to 10. The third page, when included, varied 
in content: in one survey principals were asked to indicate the importance 
of each of the 14 rating scales, and in another survey principals were asked 
about the importance of various reasons for hiring* the teacher instead of 
another candidate. 

\ 

Results 

In the three years in which the survey has been conducted, response 
rates have ranged from 947. to 977.. One mail follow-up has been used each 
year with *rom 157. to 297. of the population b$ing sent a second letter and 
questionnaire. Telephone calls have, on occasion, been placed to school 
principals in the two systems in which the principals were identified, but 
no more than three such calls were made. 

Discussion ^ 

The procedures used have been satisfactory in producing a high response 
rate in each of three administrations of the survey with relatively low levels 
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of time and money being required because -it is a mail survey utilizing only 

one follow-up. Several of the procedures are consistent with thos- associa- 

c 

ted with high response rates while others are not. 

VThe population surveyed, school administrators, is one of professionals <f 
characterized by high levels of literacy, qualities wKich have been associa- 
ted with a tendency to return questionnaires (Fowler, 1984; Sudman & Bradfrurn, 
1984). Return rates of 717. to 88% are npt unusual for teachers (Sudman & 
Bradham, 1984, p. 34). University sponsorship is also conducive to high . 
return rates (Baumgartner & Heberlein, 1984, p. 66; Sudman & Bradburn, 1984, 
p. 36). 

Principals are asked »to rate the teachers near the end of the school 

j 

year. The timing is probably advantageous because school administrators 

may have already had to perform personnel evaluations for the system (Altschuld 

& Lower, 1984). 

A minimum amount of time is required to complete the questionnaire since 
the questionnaire is relatively short and there are no open-ended items. 
In addition, confidentiality is assured. Both conditions have been associa- 
ted with positive effects (Altschuld & Lower, 1984). In some cases,- the 
respondent's identity remains unknown to the College. 

Other factors conducive to high return ratss which are characteristic 
of this survey are: personalization and follow-up mailing (Dillman, Dillman, 
& Makela, 1984; Yu & Cooper, 1983); use of multiple cover letters and central 
office support (Altschuld & Lower, 1984, p. 11); high quality paper, inside 
address added, address typed onto envelopes on both initial and follow-up 
mailings, and typed follow-up letter (Dillman, Dillman & Makela, 1984, p. 
55-56). Salience, or importance of questionnaire contents to respondents 
(Altschuld & Lower, 1984^ would have to be inferred but may be a factor since 



improving the preparation of teacher candidates would .ultimately benefit 
the principals in the future. There may be another element somewhat related 
to both salience and the nature of the population in that requesting the 
principal to do the rating accords'* recognition of his experience and judg- 
ment. 

Procedures used which are contrary to Existing research or theory in 
producing high response rates are: use of mail survey rather than telephone 
or personal interview methods (Al^tschuld & Lower, 1984; Yu & Cooper, 1983); 
lack of preliminary notification, absence of offer of premiums or rewards, 
promised or prepaid incentives (Yu & Cooper, 1983); letters not individually 
signed in blue ink, cover letters not typed originals, use of business reply, 
envelope rather than stamped return envelope, conventionl size of stationery, 
fo"llow-up sent three to four weeks after first mailing instead of sooner, 
follow-up in letter rather than postcard format, follow-up letter not hand 
signed (Dillman, Dillman, & Makela, 1984, pp. 55-56). 

Much of the research on survey methodology has. been conducted in regard 
to public opinion polls or attitudinal marketing research. It is possible 
that the nature of this survey makes many of those findings inapplicable 
to follow-up surveys. Berdie and Anderson (1974) suggest using "varying 
tactics for different samples and studies" (p. 53). It could well be that 
new answers must be sought for specific situations rather than assuming the 
generalizability of current research findings, many of which are already 
plagued by inconsistencies, such as in the areas of questionnaire length 
and class of postage (Bauagartner & Heberlein, 1984). Therefore, no claim 
is made as to the applicability of these procedures to other populations or 
types of surveys. Other researchers are encouraged to adopt or adapt such 
elements of the design as seem appropriate for their particular situations. 
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APPENDIX 

TEACHER EDUCATION FOLLOW-UP SURVEY - PRINCIPALS 

BUREAU OF EDUCATIONAL RESEARCH AND SERVICE 
College of Education 
The University of Tennessee, Knoxville 



eTKefc^efuSf: J l *™™*>»* " each item on this questionnaire. Read 

' to oarenttese f ItiJ? ™?^™ res ? onses as Seated, ignore the numbed 
in parentheses, these are for^eypunching purposes only. 

§I£Im l ' m S in iC lQS S , IV inforaation about TEACHER who graduated from 
uin m x?b>-s3 whose name appears on ^he cover letter. 

A. Teacher's level of certification: 



(cc<4) 



1. Kindergarten thjough grade 3 

2. Grades 1-8 

3. . .Grades 7-12 

4. Grades 9-12 

; 5. Grades 1-12 
6 » Ottier (Please Specify) 



B. Area (or areas) in which teacher is currently teaching: « <cc5~10) 

J* Kindergarten n . Rome Economics 

— J* ^ e f-ff ry 12 ' industrial Arts 

Agriculture 13 . Mathematics 

4. Art 



14. Music 



5. Business 15< Physical Educa tion ■ 

6. Dear Education 16 . Science 

7. Distributive Ed. — 17 . Social Studies 

!' f' Sli : S t f 18. Special Education 

10 He"Jh Lan Suage 19 .« Other (Please soecify) 



C * Jeacllin? 3 ^ 6 " CSrtified in the area °* «eas in which he/she is currently 

(cell) 

i 1. Yes 

2. Ho 

3 « Certified in one or more but not in all 

D. List atea in which teacher is teaching but not certified ; ' (ccl2-15) 

E. Reason(s) .for lack of certification in teaching area (check one or more): ' 

— l ' lilltuln. ^ ° btainin? ce "ification from State Department of ( tc 16) 

2. Still completing certification requirements frrl7 ^ 

3 - Inadequate undergraduate, advising ' / r<t( 

4. Teacher certified i n only one area tH\ 

* 5. Uncertified area is not the major teaching assignment c c2 0 

for the teacher veczuj 

6. Difficult (or unable) to obtain certified teacher due to ( cc 21) 

geographic location UcZi; 
7. Other 

. (cc22) 

F. Teacher's gender: 

(cc23) 

. 1. Male 

ERIC — z ' Fenale r 12 

10 s , 



^911^11. ^The following teacher characteristics are believed co have an influence 
reorlsenMn 8 , e££ectlv « ftM - SUSll one number for each characteristic ^ 

representing your rating of the^ teacher compared with other first-year teachers 

Sarac^is^c ^nT^ * ° f ,,GOOd,, ° n the SCale indicates that the 

ITZttu f J ^ 8 concid . £red is acceptable and that the teacher is performing 
effectively. The numbers correspond to the following scale descriptions: 



SUPERIOR: 

Reveals qualities 
found in only the 
most effective 
and creative 
teachers . 



VERY GOOD: 

Clearly above 
average. Goes 
well beyond 
meeting basic 
requirements. 

3 4 



GOOD: 

Acceptable, 
respectable 
performance. 
Meets basic 
requirements, 



MARGINAL: 

Acceptability 
only marginal; 
needs improve- 
ment. 



UNACCEPTABLE: 

Below minimum 
standards for 
certification. 
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A* Description of teacher 1 s teaching personality* 
Proper perspective of teaching-learning situa- 
tion is' maintained by teacher: teacher remains 
open and flexible. 1 2 3 4 5 6 7 8 9 10 (cc24) 

B. Apparent health and vitality. 1 2 3 4 5 6 7 8 9 10 (cc25) 

C Voice and language usage. 1 2 3 4 5 6 7 8 9 10 (cc26) 

D. Ability to work cooperatively with 

other staff members. 1 2 3 4 5 6 7 8 9 10 (cc27) 

E. Ability to profit from feedback supplied ' 

by observational techniques. 1 2 3 4 5 6 7 8 9 10 (cc28) 

F. Ability to make realistic decisions; has 

maturity of judgment. 1 2 *, 4 5 6 7 8 9 10 (cc29) 

G. Has an understanding and a working knowledge 
of content in teaching areas; understands 

.underlying principle 1 2 3 4 5 6 7 8 9 10 (cc30) 

H. Competence in effectively preparing for 

leSSOnS * 1 2 3 4 5 6 7 8 9 10 (cc3l) 

I. Classroom Management 1 2 3 4 5 



6 7 8 9 10 (cc32) 



J. Ability to perform a variety of critical 

teaching tasks as identified by analysis of 
teaching techniques (leading discussions, 

giving directions, etc.). 1 2 3 4 5 6 7 8 9 10 (cc33) 

K. Competency in evaluating students (use 

of teacher-made tests, reasonable standards 

of performance, etc.). ^ 1 2 3 4 5 6 7 8 9 10 ( cc 34) 

L. Ability to motivate learners. 1 2 3 4 5 6 7 8 9 10 Ccc35) 

M. Description of teacher's attitudes 

toward-chi-ldren-. Teacher is fair and * > * " 

just in dealing with pupils; is responsive 
to pupils and takes a "clinical 11 approach 

to misbehavior. 1 2, 3 4 5 6 7. 8 9 10 (cc36) 

N. Comnitment to teaching. J 3 1 2 3 4 5 6 7 8 9 10 ( cc37) 



(cc38) 

(cc39) 
(cc40) 
(cc41) 
(cc42) 
(cc43) 
(cc44) 
(cc45) 
(cc46) 
(cc47) 
(cc48) 
(cc49) 

(cc50) 
(cc51) 



SECTION HI . What were the major factors in your decision to hire this person 
rather than someone else who applied for the same position? (check all which 
apply.; 

__ A. Prior experience with the person during his/hei undergraduate 

program 

B - Strength of academic record 

c * Strength of recommendations 

D. Strength of student teaching evaluation 

E. Strength of test scores (National Teachers Examination) 

— __ F. Recommendation of someone within your school 

G. Recommendation of someone within your school system 

H. Personal knowledge of the person or family as community members 

I« Personal appearance 

Ability to communicate during interview 

K * Ability to communicate in writing 

L. Other 



Is there one or more factors which were critical in choosing this person? 

!• Yes 2. No 

If so, which oner's? 



Comments 
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